1. Technical Field
The present invention relates to a temporal noise reduction method and a temporal noise reduction device, which reduce noise of an image.
2. Related Art
In general, noise is added in the course of acquiring an image in an image sensor, transmitting the acquired image information, and the like. Accordingly, various noise reduction techniques such as a spatial (2D) noise reduction technique, a temporal noise reduction technique, and a spatial-temporal (3D) noise reduction technique in which the temporal noise reduction technique and the spatial noise reduction technique are combined have been developed to reduce the added noise. The spatial noise reduction technique is a technique of reducing noise using a high correlation between neighboring pixels in an image, and the temporal noise reduction technique is a technique of reducing noise using a high correlation between neighboring frames.
The prior applications for reducing image noise using such noise reduction techniques include Korean Patent Application No. 2006-115422 (image noise reducing method and device) and Korean Patent Application No. 2010-7028204 (video encoder with an integrated temporal filter for reducing noise). The related techniques are also described in “Automatic 2-D and 3-D noise filtering for television receivers”, G. de Haan, T. G. Kwaaitaal-Spassova, and O. A. Ojo, Proc. Int. Workshop HDTV '94, October 1994.
The noise added in the course of acquiring and transmitting image information serves as a factor for deteriorating image quality and also serves as a factor for deteriorating performance of image compressing techniques for transmitting data and deteriorating performance of post-processing techniques such as a recognition process.
Accordingly, in order to prevent deterioration in image quality and to improve performance of post-processing techniques, it is necessary to reduce noise.
In the temporal noise reduction techniques according to the related art, noise reduction is performed on the basis of Expression 1.FT(t)=λ·FS(t)+(1−λ)·FF(t−TP)  Expression 1
In Expression 1, FT(t) represents a noise-reduced pixel value, λ represents a filter control parameter, FS(t) represents a noise-present pixel value, and FF(t−TP) represents a pixel value located at the same position in a previously-filtered frame separated by time TP.
Referring to Expression 1, the performance varies depending on the filter control parameter λ. In general, as a motion becomes larger, λ becomes closer to 1 and the intensity of the filter becomes weaker. As a motion becomes smaller, the correlation in the time direction becomes larger and the intensity of the filter becomes stronger.
Many studies have been carried out to accurately calculate λ which is the most important factor in filter performance. A representative method of calculating λ is expressed by Expression 2.
                    λ        =                  {                                                                                                                                        a                        0                                            ,                                                                                                                                                  e                                                                    ≤                                              TH                        0                                                                                                                                                                                a                        1                                            ,                                                                                                                          TH                        0                                            <                                                                      e                                                                    ≤                                              TH                        1                                                                                                                                                                                a                        2                                            ,                                                                                                                                                  e                                                                    >                                              TH                        1                                                                                                        ⁢                                                          ⁢              e                        =                                          ∑                                  y                  =                                                            -                      M                                        /                    2                                                                    M                  /                  2                                            ⁢                                                          ⁢                                                ∑                                      x                    =                                                                  -                        N                                            /                      2                                                                            N                    /                    2                                                  ⁢                                                                  ⁢                                                                                                                      F                                                  (                                                                                    x                              +                              n                                                        ,                                                          y                              +                              m                                                                                )                                                                    ⁡                                              (                        t                        )                                                              -                                                                  F                                                  (                                                                                    x                              +                              n                                                        ,                                                          y                              +                              m                                                                                )                                                                    ⁡                                              (                                                  t                          -                                                      T                            p                                                                          )                                                                                                                                                                      Expression        ⁢                                  ⁢        2            
In Expression 2, a0, a1, a2, TH0, and TH1 are user-defined values which are arbitrarily defined by a user, and e means the SAD (Sum of Absolute Difference) of pixel values. Specifically, F(x,y)(t) represents a pixel value of a coordinate (x, y) of a frame of time t, N and M represent values corresponding to N×M which is the size of an array, and TP represents the distance between frames to be calculated. That is, e represents the absolute difference between an N×M array centered on the position of the pixel value from which noise should be reduced and an array located at the same position in a frame separated by TP.
A method of designating the user-defined values such as a0, a1, a2, TH0, and TH1 and the use thereof will be described below in brief. Since noise randomly appears in an image, a temporal filter is a filter using a technique of reducing noise through the use of a weighted sum of two pixel values on the assumption that when noise is added to a pixel in a current frame, a pixel located at the same position in a neighboring frame has a high probability that noise is not added thereto.
In general, the ranges of TH0 and TH1 are equal to the range of 3. For example, when the range of pixels is from 0 to 255 and the size of an array is 3×3, the range of e is from 0 to 255×9.
When a user designates 120 as TH0, designates 200 as TH1, and designates a0=0.2, a1=0.5, and a2=0.8 and the resultant value of the SAD centered on a pixel X is 100, the calculated SAD is located within the range of e=0 to TH0 and thus λ is determined to be 0.2 which is a0 by Expression 2. The noise reduction based on Expression 1 is performed using λ calculated in this way.
For example, a0, a1, and a2 are set to satisfy a relationship of a0<a1<a2. When the SAD value is small, the arrays to be compared have similar values, a motion is determined not to be present, a higher weight value is applied to F(x,y)(t−TP) at the time of mixing two pixels so as to form a pixel value more similar to F(x,y)(t−TP). On the contrary, when a motion is large, the SAD has a large value and a lower weight value is applied to F(x,y)(t−TP) so as not to greatly change the current pixel value.
The temporal filter according to the related art using the above-mentioned method adjusts the intensity of a filter in consideration of a motion of an image, but has a problem in that an afterimage is formed in a motion-present area to deteriorate image quality. Particularly, when the above-mentioned method is applied to a camera not fixed such as a general camera, there is a problem in that an afterimage phenomenon occurs in the whole image and image quality is greatly deteriorated.
In the temporal noise reduction technique according to the related art, filtering is performed on the whole image. Accordingly, when an erroneous value of e is set, there is a problem in that a phenomenon of mixing noise to a normal pixel having no noise occurs.
Therefore, there is a need for a temporal noise reduction technique capable of reducing a motion blurring phenomenon occurring in the temporal filter according to the related art.